A data center is a facility that houses computer systems and various networking storage, and other related components. Data centers may, for example, provide computing services to businesses and individuals as a remote computing service or provide infrastructure, platforms, or software as a service. To facilitate the utilization of data center resources, virtualization technologies may allow a single physical computing machine to host one or more instances of virtual machines that appear and operate as independent computer machines to a connected computer user. With virtualization, the single physical computing device can create, maintain, or delete multiple virtual machines in a dynamic manner.
In a large distributed computing system (e.g., multiple distributed data centers) of a computing resource service provider, various services and resources of the computing resource service provider are frequently utilized to execute a variety of different operations and/or applications. Balancing customer access to the computing resources and keeping the system secure becomes more challenging as the complexity and usage of the system increases. In such distributed computing systems, it can be difficult to manage utilization of the systems. Furthermore, it can also be difficult to isolate and troubleshoot issues with the systems, and even when issues are identified, reliably distributing the resources of the systems may be challenging. Providing remote services in a cost-effective manner can be challenging. For instance, a computing resource service provider can use the same hardware to support multiple resources.
While the hardware may be able to support average resource usage demand of multiple customers, there can be points at which overloading occurs, such as when multiple virtual machines concurrently increase utilization of limited hardware resources. In general, a computing resource, such as a physical host, is considered to be in an “overloaded” state if it is not able to provide a defined level of capacity for at least some measure of capacity (e.g., CPU cycles, network bandwidth, latency, etc.) for at least some portion of customer requests it receives. Furthermore, different hardware and software configurations of the computing resources used to provide remote program execution and remote data storage may affect performance and cause degradation of the computing resources during intervals of heavy utilization.